User interface for medical imaging including improved pan-zoom control

ABSTRACT

An ultrasonic image scanning system for scanning an organic object that includes a display system for displaying a scanned image of the organic object in a plurality of display modes wherein the display system further includes a user input device sensor for sensing a motion of a user input device in at least one of display modes to automatically switch from one display mode to a different display mode. Furthermore, the display system further counts a length of time the user input device staying idle in at least one of the display modes for switching the display mode back to a last display mode at a predefined idle time limit.

This Application is a Formal Application and claims a Priority FilingDate of Oct. 30, 2004 benefited from a previously filed Application60/623,616 filed previously by the inventor of this Patent Application.

FIELD OF THE INVENTION

This invention generally relates to system and method for carrying out amedical imaging process. More particularly, this invention relates to animproved imaging display system and method for providing more convenientuser control of the pan zoom image display for medical image systems.

BACKGROUND OF THE INVENTION

Even though there are great advancements made in the image displaytechnologies such that scanning images can be displayed with higherresolutions and can be viewed from different angles and zooming in andout at different zooming proportions to perform more accurate diagnoses.However, the user interface for displaying and viewing the medicalimages are still limited by the requirement to toggle back and forthbetween the original image and the zoomed images. These toggle back andforth operations are required when a user needs to view zoomed image ofdifferent areas of the original image. Such operations generally requirean image display user to enter many key stokes. Therefore, it becomesinconvenient for a viewer who needs to view scanned images in differentsizes.

In a conventional ultrasound imaging system, there is a dedicated ‘Zoom’key in the user interface keyboard to toggle the display in between theoriginal (un-zoomed) image and the zoom-in image. When acquiring a realtime ultrasound 2-D image on the screen, the user can press the ‘Zoom’key to highlight a graphic boarder called Zoom Region Of Interest (ZROI)for zooming the image, and then use the trackball or arrow key to panthe ZROI on the original un-zoom screen, and press the ‘Zoom’ key onemore time to display the zoom-in image. When the user wants to pan theZROI to another portion of the display, he/she needs to press the Zoomkey again to go back to the original un-zoom image and then pan the zoombox, and press the zoom key again to see the zoom-in image.

For these reasons, a need still exists for those of ordinary skill inthe art to provide an improved method and system for medical imagedisplay. Specifically, it is desirable to provide an improved userinterface to simplify the operations when a user of the image displaysystem needs to magnify different areas in an image. Specifically, it isdesirable that the number of key strokes can be reduced by automatingtoggles of display images between the ZROI zoomed-in display and the“zoomed-out” global images such that a user does not have to enter somany key strokes to change from one mode of viewing and display toanother.

SUMMARY OF THE INVENTION

It is an aspect of the present invention to provide a user interface forapplying to a medical image viewing system with enhanced image viewingcontrol to reduce the operational steps required by a viewer to switchthe zooming of different areas such that the above discussed limitationsand difficulties may be resolved.

In another aspect, the present invention provides a medical scan imagedisplay system with an anticipatory display-mode switch function thatanticipate a viewer's alternative viewing mode for the purpose ofreducing the operational steps required to change from one viewing modeto another viewing mode back and forth.

In yet another aspect, the present invention further provides animproved medical image viewing system that provides at least a globalviewing mode to view the entire scan image and a zooming mode forviewing a zoomed area on the display system. The improved image viewsystem anticipates that as a user takes certain action through an inputdevice, the user's intend is to switch from one viewing mode to anotherviewing mode and resume back to the original mode. Therefore, theimproved medical image viewing system provides automated viewing modechange functions to reduce the processing steps required by the user tochange from one viewing mode to another viewing mode.

In yet another aspect, the present invention provides a method forautomatically switch from a viewing mode, e.g. a zoomed image viewingmode, to a global viewing mode once a motion is sensed on a userinterface device such as a computer mouse, trackball, key board, or anyinputting device. The medical image viewing system therefore anticipatesthat as a viewer complete viewing a zoomed image on the display screen,the next logic step is most likely to switch back to the global imageand to select a different area for another zoomed image inspection.

In yet another aspect, the present invention provides a method forautomatically switch from a viewing mode, e.g. a global image viewingmode, to a zoomed viewing mode once a panned area is defined by a viewerthrough a user interface device such as a computer mouse, trackball, orkey board. The medical image viewing system anticipates that as a viewercompletes viewing a global image and now confirms a panned area; thenext logic step is most likely to switch back to the zoomed imageviewing for the panned area as it has completely defined now.

These and other objects and advantages of the present invention will nodoubt become obvious to those of ordinary skill in the art after havingread the following detailed description of the preferred embodiment,which is illustrated in the various drawing figures.

BRIEF DESCRIPTION OF FIGURES

The present invention is described in detail below with reference to thefollowing Figures.

FIG. 1 is a diagram for showing the two viewing modes of an imagedisplay system where a zoomed image display represents an amplifiedimage of a panned area on a global image.

FIGS. 2A to 2G are a series of images showing the automatic switchingoperations provided by an improved imaged display system of thisinvention.

FIG. 3 shows images for illustrating different operational sequences ofanother preferred embodiment of an ultrasonic image scanning system ofthis invention.

FIG. 4 shows a display system to simultaneously display a zoomed-outimage, as the upper image, and a zoomed-in image as the lower image,with different screen layouts.

FIG. 5 shows a display system further switches automatically from afirst display mode, e.g., a global B or B/Color image shown as the topimage, to a second display mode, e.g., a blood flow spectrum of aspecific vessel on the top image as shown on the bottom image.

DESCRIPTION OF PREFERRED EMBODIMENTS

Medical imaging systems are known. For example, ultrasound imaging isdiscussed in the '080 patent and in U.S. Pat. No. 6,248,071. Thedisclosures made in that Patent are hereby incorporated by reference inits entirety for all purposes in this Patent Application. As illustratedin FIG. 1, a medical image display system is frequently toggled back andforth between a global view and zoom of an image as shown on the leftside of FIG. 1. Then with a panned area, a zoom image is shown with amagnified image of the panned area. The zoomed image as shown on theright hand side of FIG. 1 is often required to carry out detailinspection to determine if any abnormal or unusual characteristics aredetected in the scanned image for the purpose of carrying out adiagnosis.

Referring to FIGS. 2A to 2F for the operational sequences of a preferredembodiment. In FIG. 2A, an original image is shown on the screen and inFIG. 2B, a user presses a zoom key to bring out the ZROI graphic andthen resize and/or pan the ZROI box with a track ball on a user selectarea. In FIG. 3C, as soon as the panning action is completed, a zoomedimage is shown on the screen, therefore a separate keystroke to show thezoomed image is not required. In FIG. 3D, as soon as the user touchesthe trackball or the keyboard, an original image is shown automatically.Another sets of keystrokes are therefore not necessary to end the zoomview to switch back to the original global view. In FIG. 3E, the usercan again pan the ZROI graphic box to a different area, and then in FIG.2F, the display switch to a zoomed image when the ZROI window stays atthe same location (i.e., if trackball is stationary) for a certainlength of time (e.g. 0.3 second) thus saving additional keystrokes thatare generally required to switch from a global view to a zoomed view.

Referring to FIG. 3 for different operational sequences of anotherpreferred embodiment of an ultrasonic image scanning system of thisinvention. The image scanning system includes a display system thatsimultaneously displays at least two images with a first image as shownin the upper image that displays the tissue and blood vessels, generallyreferred to as a “B-image with 2-D brightness”. The display systemfurther display a second image, e.g., a lower image, that shows a crosssectional view of the organic object cut off at a scrolling line, e.g.,line A-A′, across the first image, generally referred to as a M-imagedisplayed with motion scrolling mode. In the normal operation, theM-image is updated in real time and the B-image is in frozen state. Atthe moment the user touch the trackball, the display switches to B-imagein real time and M in frozen mode. Now the user can use the trackball toalter the A-A′ on the B-image for M-image display. As soon as thetrackball stop (>0.3 sec for instance), the display will switch toM-image in real time and B in frozen. Referring to FIG. 4 for anotherpreferred embodiment, wherein the display system further simultaneouslydisplays a zoomed-out image, e.g., the upper image, and a zoomed-inimage, e.g., the lower image, with different screen layouts. In normaloperation, the Zoom-in image is in real time update and the zoom-outimage is in frozen state. When the user touches the trackball and usesit to pan and/or resize the ZROI, the zoom-out image is automaticallyactivated. The zoom-in image resumes to the real time update mode whenthe trackball stops for less than 0.3 sec for instance.

Referring to FIG. 5 for another preferred embodiment, wherein thedisplay system further switches automatically from a first display mode,e.g., a global B or B/Color image shown as the top image, to a seconddisplay mode, e.g., a blood flow spectrum of a specific vessel on thetop image as shown on the bottom image. The second display mode, e.g.,the image for showing the blood flow spectrum of a specific vessel is todisplay an image that is graphically unrelated and independent from animage displayed in the first display mode, e.g., the B or B/Color imageshown in the global view mode. In the normal display, the Spectrum modeis in real time updating and the B or B/color is in frozen mode. Whenuser touches the trackball and selects different vessel for Spectrumdisplay, the system automatically make B or B/color in real time updatemode. As soon as the trackball stops for less than 0.3 sec (forinstance), the system switches the Spectrum mode in real time, and B orB/color in frozen.

Among some embodiments of the present invention are a system and amethod capable of performing ultrasound or different method of scanningand to display these images on a viewing screen. Among some embodimentsof the present invention are a 3-dimensional Computer Aided Diagnostic(CAD) software package that can further enhance the analyses and displayof the scanned images. In a different embodiment, the auto pan zoomfunction can be applied to ultrasound B mode (including Tissue HarmonicB), and simultaneous B/Color flow mode

Embodiments of this invention further include display system with imagesources obtained from medical scanning operations for diagnosticinspections and such image display systems includes various userinterfaces with devices and software tools to control the display indifferent angles, different amplified proportions and different colorsfor better viewing and more accurate diagnoses.

Embodiments of the present invention include implementations of softwareand hardware systems and functions for providing a user interface formedical imaging. Embodiments of the present invention also include amedical imaging method and a medical imaging system that include methodand system for providing the improved user interface.

Embodiments of the present invention further include medical scanningsystems connecting to image display systems with options for either realtime display or post scanning display. One exemplary system is anultrasound scanning system connected to an image display system withuser graphic interface and user controller to change display images forviewing at different viewing modes.

In one embodiment, the controller may include a computer system, forexample, a special-purpose or general-purpose computer that isprogrammed with software according to an embodiment of the presentinvention. The software may reside on a computer memory device fromwhich the software can be read to ultimately instruct a computer systemto perform steps. Any type of computer memory device may be used, fornonexclusive example, ROM, RAM, optical media, magnetic media, physicalmedia, and the like. Media may be of any competent form, for example,disk, tape, stick, integrated circuit, and the like. Computer systemsare well known, and any competent type may be used, for example,general-purpose, or special-purpose, or embedded, or physically orwirelessly networked, or multi-processor, or single-processor, and soforth. Any competent operating system may be used, for example,Microsoft Windows, Unix, GNU (e.g., GNU/Linux), or any other competentoperating system whatsoever.

In another embodiment, the controller is configured to be able tocommunicate at least occasionally with various portions of the remainderof the medical imaging system, e.g., via a connector or bus or networkor the like, e.g., via wires or wirelessly, according to any competentcommand structure or communication/interconnection protocols and anyoperable and properly functional protocols that may or may not becommercially available.

In another embodiment, the controller operates to produce informationfor display to a user and to accept input from the user. The displayingand inputting may be at close physical proximity to the controller'smain computer processor, or the displaying and inputting may be via aclient computer or terminal at a remote location. For example, thecontroller may be an Internet or intranet or other type of server, andthe user may interact remotely with the controller from a browser orother remote interface.

In another embodiment, there is a controller, or a method thereof, or acomputer-readable memory product that includes the software thereof,that provides a user interface that includes an improved Pan-Zoomcontrol that requires fewer keystrokes to operate than is required underthe conventional approach. Remaining portions of the controller (or itsmethod or software) can be according to any conventional system ormethod.

In a specific embodiment, the present invention provides an automaticPan-Zoom for user interface in any ultrasound system so the user canreduce the keystrokes in operating the unit. According to someembodiments, the user can easily manipulate the pan zoom window withtrackball (or mouse, or joystick, or touchpad, or cursor keys, or thelike) and know the portion of the full image corresponds to the zoomwindow. Once the user touches the trackball (or any inputting device . .. ) in the zoom-mode during real-time scanning, the screen willautomatically switch back to the original un-zoomed image with a ZROIgraphic box highlighted, once the ZROI is panned and stay at the samelocation for a certain time (e.g. 0.3 second) (e.g., a parameter withina range of about 0.2 to 1.0), the system will automatically go back tothe magnified zoom-in display, and the user does not have to hit thezoom mode key to toggle back and forth between the zoom and un-zoomedmodes. This makes it easy for the user to operate the unit and increasethe throughput rate.

In a specific embodiment, the controller (or its method or software) mayhave a soft-menu to allow the user to set the system default to eitherthe conventional pan zoom or the auto image control Pan zoom.

In a specific embodiment, the auto pan zoom function can be applied toultrasound B mode (including Tissue Harmonics B), and simultaneousB/Color flow mode. The auto zoom function has options and can be appliedto the rectangular and trapezoid image for flat linear array probe, orsector image for the curve linear array and phase array probe. Theimplementation of the zoom function can be in read zoom, which the zoomimage is interpolated from the acquired image in the memory; or thewrite zoom, which the zoom image is a new acquisition data withdifferent resolution setting.

In a specific embodiment, the auto pan zoom can be applied to theoffline ultrasound image reviewing station, in which the plural ofimages are display on the screen as slides, and the highlighted slideimage is magnified to the center of the display. There may be slidesbehind the center of the display and are block by the magnified image.When the user access to the trackball or any inputting device, thescreen will go back to the original display with all slides on thescreen, and the magnified zoom image at the center disappear. Once thetrackball stay at the same slide for a moment of time, the display willautomatically be back to the highlighted image zoom at the center of thescreen. The display can stay at ‘all slides’ image if the trackballcursor is not within any slide boarder.

With this auto pan zoom function, a soft-menu can be used to set up thedwelling time for the trackball in order for the system to decide if itneeds to resume to the zoom image screen. For some embodiments of thepresent invention, the image display systems are especially useful forimplementations in which the zoom-mode image window fills the entirephysical display screen or in which the zoom-mode image window is theonly medical image window on the physical display screen.

In another preferred embodiment, the implementation of the zoom functioncan be in read zoom, which the zoom image is interpolated from theacquired image in the memory; or the write zoom, which the zoom image isa new acquisition data with different resolution setting. Furthermore,the auto pan zoom operational processes can also be applied to theoffline ultrasound image reviewing, in which the plural of images aredisplay on the screen as slides, and the highlighted slide image ismagnified to the center of the display. There may be slides behind thecenter of the display and are block by the magnified image. When theuser access to the trackball, the screen will go back to the originaldisplay with all slides on the screen, and the magnified zoom image atthe center disappear. Once the trackball stay at the same location for amoment of time, the display will automatically be back to thehighlighted image zoom at the center of the screen.

The description and the drawings of the present document describeexamples of embodiment(s) of the present invention and also describesome exemplary optional feature(s) and/or alternative embodiment(s). Itwill be understood that the embodiments described are for the purpose ofillustration and are not intended to limit the invention specifically tothose embodiments. Rather, the invention is intended to cover all thatis included within the spirit and scope of the invention, includingalternatives, variations, modifications, equivalents, and the like.

1. An ultrasonic image scanning system for scanning an organic objectcomprising: a display system for displaying a scanned image of saidorganic object in a plurality of display modes wherein said displaysystem further comprising a user input device sensor for sensing amotion of a user input device in at least one of display modes toautomatically switch from one active display mode to a different activedisplay mode; and said display system further counts a length of timesaid user input device staying idle in at least one of said activedisplay modes for switching said active display mode back to a lastactive display mode at a predefined idle time limit.
 2. The ultrasonicimage scanning system of claim 1 wherein: said display system furthersimultaneously displays at least two images with a first image showingan external view of said organic object and a second image showing across sectional view of said organic object cut off at a scrolling lineacross said first image wherein said second image is further activatedto show a cross sectional view corresponding to a cross section as saidscrolling line move to different positions over said first image.
 3. Theultrasonic image scanning system of claim 1 wherein: said display systemfurther simultaneously displays a zoomed-in and a zoomed-out images withdifferent screen layouts.
 4. The ultrasonic image scanning system ofclaim 1 wherein: said display system further switching automaticallyfrom a said first display mode to a second display mode wherein saidsecond display mode is for displaying an image that is graphicallyunrelated and independent from an image displayed in said first displaymode.
 5. The ultrasonic image scanning system of claim 1 wherein: saiddisplay system further enabling an operator to set said predefined idletime limit for automatically switching from one of said display modes toanother one of said display modes when said user input device stayingidle.
 6. The ultrasonic image scanning system of claim 1 furthercomprising: a three dimensional (3-D) image as first scan mode and B orB/color as the second scan mode.
 7. The ultrasonic image scanning systemof claim 1 further comprising: A Spectral Doppler as first scan mode andB or B/color as the second scan mode.
 8. The ultrasonic image scanningsystem of claim 1 further comprising: a M-mode as first scan mode and Bor B/color as the second scan mode.
 9. The ultrasonic image scanningsystem of claim 1 further comprising: a Zoom-out as first scan mode andZoom-in as the second scan mode.
 10. The ultrasonic image scanningsystem of wherein: said display system further includes an automatic panzoom function for application to an ultrasound B mode and a simultaneousB/color flow mode.
 11. The ultrasonic image scanning system of claim 1further comprising: an image source for interfacing and obtaining adisplay image from different image scanning systems.
 12. The ultrasonicimage scanning system of claim 1 wherein: said display system furtherincludes an image color management function to manage a display ofimages in different colors.
 13. The ultrasonic image scanning system ofclaim 1 wherein: said display system further includes a displaymanagement software package to manage a display of images in differentangles, amplified proportions and different user interfaces.
 14. Animage display system for displaying an image in different display modescomprising: a user input device sensor for sensing a motion of a userinput device in at least one of display modes to automatically switchfrom one display mode to a different display mode; and said displaysystem further counts a length of time said user input device stayingidle in at least one of said display modes for switching said displaymode back to a last display mode at a predefined idle time limit. 15.The ultrasonic image scanning system of claim 14 wherein: said displaysystem further enabling an operator to set said predefined idle timelimit for automatically switching from said display mode to said lastdisplay mode when said user input device staying idle.
 16. The imagedisplay system of claim 14 wherein: said user input device sensorsensing a motion of a user input device in a zoomed view display mode toautomatically switch from said zoomed view display mode to a global viewdisplay mode.
 17. The ultrasonic image scanning system of claim 14wherein: said display system further counts a length of time said userinput device staying idle in a global view display mode for switchingback to a zoomed view display mode as a last display mode at apredefined idle time limit.
 18. A method for displaying an image indifferent display modes comprising: sensing a motion of a user inputdevice in at least one of display modes to automatically switch from onedisplay mode to a different display mode; and counting a length of timesaid user input device staying idle in at least one of said displaymodes for switching said display mode back to a last display mode at apredefined idle time limit.
 19. The method of claim 18 furthercomprising: enabling an operator to set said predefined idle time limitfor automatically switching from said display mode to said last displaymode when said user input device staying idle.
 20. The method of claim18 wherein: said step of sensing a motion of a user input device is astep of sensing said motion of said user input device in a zoomed viewdisplay mode to automatically switch from said zoomed view display modeto a global view display mode.
 21. The method of claim 18 wherein: saidstep of counting said length of time said user input device staying idleis a step of counting said length of time when said user input devicestaying idle in a global view display mode for switching back to azoomed view display mode as a last display mode at a predefined idletime limit.
 22. The method of claim 18 further comprising: implementinga display control software package to manage a display of images indifferent angles, amplified proportions and different user interfaces.